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ABSTRACT 

This study points out the need for evaluation 
criteria for business resources on the Internet, so that both 
corporations and special libraries will be better able to judge the 
value of the Internet as an information source and increase their 
competitive edge in today's global market. The research addresses 
concerns that unfiltered information on the Internet is rarely 
subject to standards for validity and accuracy. Data was collected 
from two rounds of ques t i onnai res sent to information specialists, 
librarians, and corporate Internet users through listservs and 
newsgroups. The respondents' comments suggested that business 
information on the Internet may well rise in perceived or actual 
reliability if it includes statements regarding the source of the 
data, if prefatory materials, "readme" files, or a statement on 
frequency of updates are added, and if the index shows a large degree 
of detail and depth. The results also stressed the need for 
evaluative guides rather than simple locators, for demonstration of 
greater usefulness of Internet information in general, and for 
filenames that better reflect information content. There was 
disagreement over whom should be called upon to index or catalog 
electronic information and whether Internet sources could eventually 
replace print sources. Sample questionnaires, with accompanying data 
for each question, are appended. (Contains 43 references.) (BEW) 
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ABSTRACT 



By developing an evaluative guide to business resources 
on the Internet, both corporations and special libraries will 
be better able to judge the value of this information source 
as well as increase their competitive edge in today's global 
market. This study combined a modified version of the Delphi 
technique with survey data collection. To obtain quantitative 
data, a questionnaire was sent along with generalized 
statements regarding the Internet . The resulting consensus 
formed from the experts illustrates the need for access to the 
Internet in addition to providing a set of solutions to 
6^9-lu3-tive concerns regarding this information resource. 
Results will provide a basis for continuing research in this 
area of information exchange while allowing its current users 
to select their sources for maximum efficiency. 
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INTRODUCTION 



1.1 BACKGROUND OF THE STUDY 

The Internet is not one specific entity, but rather a 
term given to a computer network which is comprised of 
thousands of computers linked together worldwide. Other 
networks, such as Bitnet and Usenet, although separate 
entities, are considered a part of the term "Internet" for 
this study. All these sources offer almost unlimited 
opportunities in electronic communication, information 
transfer and research. Many special libraries and businesses 
are using the network to further their information needs and 
competitive advantage. The overwhelming barrage of information 
available through the Internet, however, is at once both 
positive and negative. To date, there is no evidence of 
scientific evaluation available for information on the 
network, which is traditionally important to these user 
groups. Although generalized treatment of this area has been 
addressed through popular and scholarly literature, scientific 
research has only just begun. It is vital to establish a set 
of benchmarks against which resources available through this 
network can be measured and evaluated for quality. Expert 
consensus was sought to develop these benchmarks and provide 
a basis for conclusions regarding the value of the Internet as 
an essential element in today's expanding world economy. 

1 
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1.2 NEED FOR THE STUDY 



Popularity and exponential growth of the Internet have 
increased dramatically over the past few years, now including 
commercial users as well as universities and research 
organizations. Special libraries, while seeing the need to 
utilize the networks for resource sharing and additional 
patron services, must redefine their roles in a global 
networking environment. To this end, establishment of resource 
assessment guidelines will enable information specialists to 
provide quality services in the business world by navigating 
through an uncertain environment and selecting appropriate 
tools. The ability to locate accurate information is essential 
to businesses and libraries that provide information services 
to these users. By utilizing the results of this study, the 
Internet will be useful as an information source as well as 
help to increase productivity of information professionals. 



1.3 OBJECTIVES 



The following questions served to guide this research 
project. The intent of this study was to clarify these 
questions as v;ell as examine the current status of the 
Internet and its place in special libraries and businesses. 

1. Though directories exist, are there any evaluative 
guides to apply to business resources on the Internet? 

2. Are there any filters available to distinguish 
valid, quality data from incomplete, unverified information? 

3. Are there any groups, experts or areas on the network 
which require certain standards of quality for their data 
before it is posted, and is this data verified? 

4. What standards are important to the business and 
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special library communities with regard to their information 
needs? 

5. If information from the network is utilized by these 
user groups, does it aid them with respect to efficiency? 
Would an assurance of accurate information on the network 
increase both its efficiency and the usefulness of this 
resource? 

6. If business information is found on the network, is 
it assumed to be accurate or used without guarantees of 
validity? 

7. Are special libraries and businesses interested in a 
resource where information gained through the network is 
subjectively reviewed by experts and judged to be quality 
information before they receive it? Would they pay for this in 
addition to the regular online fee for connection to the 
network? 



1.4 DEFINITIONS OF TERMS 

For this study, the Internet, also referred to as the 
network in the study, is defined as the worldwide network of 
computers running the Transmission Control Protocol/Internet 
Protocol, or TCP/lP. This will include any smaller local 
networks which have the capability to transmit electronic data 
and connect to the Internet. Other network terms are defined 
as follows: 

FTP- -File Transfer Protocol; a protocol that defines how to 
transfer files from one computer to another.^ 

Telnet- -A "terminal emulation" protocol that allows you to log 
in to other computer systems on the Internet.^ 

^ Ed Kr o 1 , The Whole Internet: User's Guide and Catalog , 
ed. Mike Loukides, (California: O'Reilly & Associates, 1992), 
357 . 

'’Ibid. , 362 . 

3 
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Special libraries are defined as any member of the 
United States Special Libraries Association, headquartered in 
Washington, DC (see note in Appendix A for libraries classed 
by subject) , or any library associated with a business or 
institution which provides specialized collections or 
services. Businesses refers to any organization either profit 
or non-profit engaged in activities of exchanging commodities, 
meaning either goods or services. 

Information needs are defined as any data which meet a 
set of criteria established as essential or helpful by the 
user groups included in this study. No attempt was made to 
include the concept of "information" as an element to be 
examined. Research into this area was considered beyond the 
scope of this study and it is suggested that the reader 
consult other papers for more complete coverage of this 
theory. 

Evaluative guides, or rssource guidelines, refers to a 
checklist of items, either questions or statements, which can 
be consulted to judge the irtegrity and worth of any resource 
available through the Internet. This does not encompass basic 
directory information currenciy available in print or through 
the network. Evaluation will consider the following items: 

(a) credibility of source or information presented 

(b) intended audience consideration 

(c) reference value versus timeliness 

(d) presentation of material 
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(e) uniqueness of material^ 

(f) clarity and knowledge of search topic 

(g) depth of information needed 

(h) familiarity with relevant items'* 

Quality, or value, will encompass the same 
characteristics that Auster and Choo felt important from 
previous studies regarding "perceived source quality. . . which 
include relevance, reliability, accuracy, quantity, and 
timeliness . 



^Morris E. Chafetz, "Toward Increasing Quality 
Consciousness for Alcoholism Literature, " Journal of the 
American Society for Information Science 27, no. 3 (May- June 
1976) : 162-170 . 

“David R. Morehead and William B. Rouse, "Online 
Assessment of the Value of Information for Searchers of a 
Bibliographic Data Base," Information Processing and 
Management 21, no. 2 (1985): 83-101. 

^Ethel Auster and Chun Wei Choo, "Environmental Scanning 
by CEOs in Two Canadian Industries, " Journal of the American 
Society for Information Science 44, no. 4 (May 1993) : 194-203, 
citing R. Zmud, "An Empirical Investigation of the 
Dimensionality of the Concept of Information, " Decision 
Sciences 9 (1978) : 187-195; R.S. Taylor, Value-added Processes 
in Information Systems . (Norwood. N.J.: Ablex, 1986); M.S. 
Nilan, R.P.Peek and H.W. Synder, "A Methodology for Tapping 
User Evaluation Behaviors: An Exploration of User's Strategy, 
Source and Information Evaluating," in Proceedings of the 51st 
ASIS Annual Meeting, Atlanta, GA, October 23-27, 1988 . by the 
American Society for Information Science: American Society for 
Information Science, 1988, 152-159; D. Halpern and M . S . Nilan, 
"A Step Toward Shifting the Research Emphasis in Information 
Science from the System to the User: An Empirical 
Investigation of Source-evaluation Behavior, Information 
Seeking and Use," in Proceedings of the 51st ASIS Annual 
Meeting, Atlanta, GA, October 23-27, 1988 by the American 
Society for Information Science; American Society for 
Information Science, 1988, 169-175. 
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Filters will be defined by one of the following 
elements : 

(a) professional or expert opinion; or 

(b) Internet groups, such as academically monitored 
discussion groups, electronic journals or 
newsgroups . 

Efficiency will be used as Webster's New Collegiate 
Dictionary defines it, "effective operation as measured by 
comparison of production with cost (as in energy, time, and 
money) . 

1.5 LIMITATIONS OF THE STUDY 

The Internet and issues relating to it are only now 
beginning to be researched. A decided limitation to this study 
was the reliance on secondary sources such as journal 
articles, presented papers and popular literature, due to an 
absence of scientific study in this area. Research in other 
subject areas was considered to provide a basis for general 
evaluation guidelines. 

Criticism of methodology, that being a modified Delphi, 
is accepted as another limitation. This study used, as 
Linstone and Turoff explain, "experiences as the prevailing 
mode of interaction ... occurring when (a) panelists are 
familiar with each other and identify with the subject or 



^ Webster's New Collegiate Dictionary , 1975 ed . , 
"Ef f i ciency . " 
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spon.-?or of the Delphi inquiry and (b) the generic form of 
expected product of the inquiry is clearly indicted."’’ Because 
initial contact of potential experts was obtained through e- 
mail, many members shared common interests and possibly a 
knowledge of others who may have been included in the expert 
group. However, due to the nature of the Internet and its 
communication flexibility, the results of this study can be 
generalized to a wider user population than studied. Concerns 
over choice of experts is accepted as bias to an extent, and 
the researcher encourages further investigations to increase 
the validity of the findings. 



2 . LITERATURE REVIEW 

Extensive review of relevant databases was conducted, 
including the following: ERIC, Information Science Abstracts, 
Library Literature, Lisa, Infotrac, Sociofile, Psychlit, 
Dissertation Abstracts and Dialog. Online Public Access 
Catalogs used included Kent State's Catalyst and Ohiolink. 
Electronic resources, such as e-mail, newsgroups and online 
sources were consulted. 

Some of the literature reviewed is not actual research, 
which is a result of the recent commercial development of the 
Internet and as of this date, no available research. Expansion 
into related literature, such as techniques of information 

^Harold A. Linstone and Murray Turoff, ed. The Delphi 
Method (Reading, Mass.: Addison-Wesley Publishing Co . , 1975), 
59 . 
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retrieval, value and relevance of information, users' needs 
and information seeking behaviors was utilized. 

Earliest literature helped to develop a basis for 
evaluating network information. Chafetz in "Toward Increased 
Quality Consciousness for Alcoholism Literature" provided some 
considerations for evaluation. His comments referred to print 
sources, yet credibility and intended audience, among other 
criteria, are considerations which need to be addressed in a 
network environment. Censorship through evaluation was another 
concern which Chafecz® and the researcher consider 
inappropriate and avoidable. In 1978, Zmud offered insight 
into the evaluation elements for management information 
systems. He established essential elements needed for 
information evaluation within the context of a users' 
perception. These suggested dimensions of information have 
continued to be the standards upon which further research have 
been based. His conclusion that ". . .the findings reinforce the 
sentiment that the manner in which data is presented to a 
decision maker is critical to his perception and subsequent 
usage of the data..."^ was considered relevant for this study. 
The research completed by Morehead and Rouse contributed 
further elements for consideration, based partially on the 
findings of Zmud. Of these, perhaps the four concerns of 

®Chafetz , 164 , 170 . 

^Robert W. Zmud, "An Empirical Investigation of the 
Dimensionality of the Concept of Information, " De< ision 
Sciences 9, no. 2 (1978): 187-195. 
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knowledge, depth and familiarity are the most 
to network information seeking.^® A study on 
services, information and evaluation by Deutsch and Malmhorg 
investigated the issues of stakeholders and . evaluatic . 
Although weak in its limitations and conclusions, this report 
did reinforce the concept that a user's perception, or in this 
case, the stakeholder's objectives, could affect how 
information or services were judged. 

In 1991 , in "Beyond OPACS . . .The Wealth of Information 
Resources on the Internet," Kalin and Tennant pointed out, 
even at this early stage of the Internet, the problems with 
evaluating electronic information. The " . . .inability to browse 
quickly the contents is still an issue today and it is 
evident that judging the contents of information resources on 
the network will require different techniques than those used 
for print or CD-ROM. Caroline Arms continued this thought in 
"Internetworking Services and the Electronic Library," by 
commenting on the lack of centralized control of services and 



’■^Morehead and Rouse, 88. 

'^Stuart Jay Deutsch and Charles J. Malmborg, "A Study of 
the Consistency of Stakeholder Preferences for Different Types 
of Information in Evaluating Police Services," Evaluation and 
Program Planning 9, no. 1 (1986): 13-24. 

'^Sally W. Kalin and Roy Tennant, "Beyond OPACS... The 
Wealth of Information Resources on the Internet, " Database 14, 
no. 4 (August 1991); 28-33. 
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the problems identifying valuable resources.'-^ The collection 
of independent groups contributing to the network is at once 
both a strength and a weakness, depending on the viewpoint one 
chooses . 

Nineteen-ninety-two brought about more detailed 
discussions regarding the growth and possibilities associated 
with the network. Special librarians were already using the 
Internet to network with each other and for resource sharing, 
according to a survey by Ladner and Tillman. Over ninety 
percent of these users responded that e-mail was the primary 
use and major advantage of the Internet. The use of the 
network for improved communication through e-mail was also 
found to be a benefit to Dynix corporation, in "Using 
Internet: Benefits to a Corporation." They included increased 
productivity and fostering professional development as 
additional positive factors. Research and discussion 
dealing with information and its value continued to be 
explored. Of the discussions, Duncan continued previous 
research observations in "The Essentialities of Productivity 

“David H. Brunell ," Internetworking Services and the 
Electronic Library," Journal of Library Administratio n 15, no. 
3/4 (1991) : 21-36, citing Caroline R. Arms, "A New Information 
Infrastructure," Online 14, no. 5 (September 1990): p.l9. 

“Sharyn J. Ladner and Hope N. Tillman, "How Special 
Librarians Really Use the Internet," Canadian Library Journal 
49, no. 3 (June 1992): 211-215. 

'■’’Gail Wanner, "Using Internet: Benefits to a 
Corporation, " in 13th National Online Meeting: Proceedings 
1992 . by the National Online Meeting (Medford, NJ: Learned 
Information, 1992), 417-426. 
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in Information Services, " by stating that it is the users who 
define the value of information. His ideas t.hat information is 
subjective in nature, time dependent and situation 
dependent*^® are concepts which were considered when 
developing the questionnaire used in this study. Clark and 
Augustine also researched the measurement of information 
value. Their findings contributed important evidence that 
inferior information quality, and specifically a decrease in 
accuracy, affected a firm's profitability, cost performance 
and efficiency.^’' 

Allan Earle in "Hacker Heaven" discussed the exponential 
growth of the Internet in 1993, listing estimates of over 
twenty million users in ninety-one countries. The number of 
corporate networks linked to the network was estimated at over 
fifteen thousand, and the current rate of expansion placed 
estimates that the network was "doubling in size every ten 
months ... creating one of the pivotal business developments of 
the nineties."^® Perhaps his most relevant point was noting 
that with ease of access to the networks, the size of a 
business was no longer a concern. Smaller enterprises were 



'^Joseph W. Duncan, "The Essentialities of Productivity 
in Information Services," Information Society 8, no. 2 (April- 
June 1992) : 77-82 . 

^’Thomas D. Clark and Fred K. Augustine, Jr., "Using System 
Dynamics to Measure the Value of Information in a Business 
Firm," System Dynamics Review 8, no. 2 (Summer 1992) : 149-173. 

'“Allan Earle, "Hacker Heaven, " Canadian Business 66, no. 
12 (Dec. 1993) : 63-65 . 
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able to compete in a global market as well as the large 
corporations. The president of the Internet Society felt that 
the network could reach estimates of one billion network 
systems, while a member of the International Engineering Task 
Force noted the problems with data integrity still remained, 
indicating continued growth with little attention to 
quality.^® Many of the journal articles commented on the 
continued importance of the networks for e-mail and its 
usefulness in the collaborative efforts of businesses and 
scientific studies. Relevant papers published during this year 
included more cognitive research on information value and 
behaviors. One study, by Auster and Choo, while refuting 
earlier research in user studies, did correlate with the 
findings of the previously discussed 1992 study from Clark and 
Augustine. The conclusions from Auster and Choo's 
investigation into environmental scanning patterns of business 
CEOs found that of "environmental uncertainty, source 
accessibility, and source quality, source quality is the most 
important factor on explaining source use in scanning."^® 
Results from Rocheleau's "Evaluating Public Sector Information 
Systems: Satisfaction versus Impact" discussed many concepts 
relating to information systems which apply to the network 
environment. When evaluating user satisfaction with 



*®Cheryl Gerber, "Booming Commercial Use Changes Face of 
Internet," Inf oworld 15, no. 15 (April 1993): 1, 38. 

^®Auster and Choo, 194. 




13 



information retrieval and search strategies, he found that 
many times users' expectations of information systems are too 
high, thus creating dissatisfaction when reality does not 
match their goals. Related to this was the conclusion that 
user perceptions of a system were often more important than 
reality. He stated that "if users don't think that an 
information system is having any impact, they may be less 
likely to use or support the system. Additionally, many 
times decision makers may benefit from information obtained 
through an information system, yet they may not actually 
utilize the system themselves. His observations that 
satisfaction is dynamic, changing as the user learns, support 
earlier studies in this area. This indicates the need fc : a 
distinction between user satisfaction and impact of the 
information obtained on the organization's goals and 
performance to preserve a valid evaluation of both entities. 
Satisfaction has often been judged through relevance, and the 
amount of research in this area is immense. The intent of this 
study was not to continue relevance research, but to 
incorporate it as one element to be considered when evaluating 
network resources. 

An investigation into relevance by Louise T. Su found 
^ that it should not be the only criterion for system 
evaluation. She included effectiveness, efficiency, 

“‘Bruce Rocheleau, "Evaluating Public Sector Information 
Systems: Satisfaction Versus Impact," Evaluation and Program 
planning 16, no. 2 (April-June 1993): 119-129. 
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interaction, searcher, service, system and user as factors 
which should be considered when attempting to evaluate an 
information retrieval system. These factors were 

incorporated into this study by combining them with other 
elements in the initial Delphi questionnaire. Document 
presentation and representation on the networks were 
additional areas which were included in the evaluation 
criterian for this study. The Online Computing Library Center 
(OCLC) , located in Ohio, completed research involving the 
access, use, and cataloging of information found on the 
Internet. Of all the research available, this study was 
perhaps the most vital. Their findings indicated several 
important areas which needed to be incorporated into a 
consideration of presentation and evaluation of materials. 
They concluded that most traditional methods of identification 
and cataloging of materials were not adequate for Internet 
information. Most files contained incomplete bibliographic 
details, information was difficult to locate, and the "value 
of the information files such as 'readme' or 'index' could 
vary greatly depending on the completeness, clarity, and 



^^Louise T. Su, "Is Relevance an Adequate Criterion for 
Retrieval System Evaluation: An Empirical Inquiry into the 
User's Evaluation," in ASIS ' 93 : Proceedings of the 56th ASIS 
Annual Meeting, Columbus, Oh, October 24-28, 1993 , by t he 
American Society for Information Science (Medford, New Jersey: 
Learned Information, Inc., 1993), 93-103. 
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currency of the descriptive information provided. There 
were indications these files could be more useful, yet they 
occurred infrequently. Another problem was the low correlation 
between file contents and file names. The amount of files 
stored at any site capable of providing FTP services was also 
found to create a difference in usefulness. The larger FTP 
sites tended to have more directory hierarchies, thus 
organizing their information better than sites with a smaller 
number of files. Carol Barry, in her research on document 
representations and predicting relevance, found that abstracts 
and titles were used to indicate relevance and provide the 
most information regarding an item.^"* Her conclusions, 
combined with the OCLC findings, indicate some of the 
obstacles librarians and businesses face in identifying, 
obtaining and evaluating quality items through the Internet. 

It is evident that users of information have been 
studied for their preferences and needs. Quality of 
information has been demonstrated to be a primary factor in 
the usefulness of information to businesses and libraries. 
Through a consensus of experts, evaluative guidelines can be 



^’Martin Dillon, Assessing Information on the Internet 
(Dublin, Oh: OCLC Online Computer Library Center, Inc., Office 
of Research, 1993), 1-39, OCLC, OCLC-OR-RR-93-1 . 

^“Carol L. Barry, "A Preliminary Examination of Clues to 
Relevance Criteria within Document Representations, " in ASIS 
'93: Proceedings of the 56th ASIS Annual Meeting, Columbus. 

Oh, October 24-28, 1993 , by the American Society for 

Information Science (Medford, New Jersey: Learned Information, 
Inc., 1993), 81-86. 
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established which will enable these users to identify and use 
the best of what the Internet has to offer. 



3. PROCEDURE 

3.1 METHODOLOGY 

A modified version of the Delphi technique was used for 
this study. The tool used was a series of questionnaires 
designed to extract expert opinions regarding development of 
evaluation standards for Internet resources. Initial 

contact of potential experts was accomplished by posting a 
notice to discussion and Usenet groups. Initially, it was 
planned to send questionnaires to members of the Special 
Libraries Association in northeast Ohio, as well as randomly 
selected corporations. After receiving well over fifty 
inquiries through e-mail, the final Delphi group consisted 
primarily of information specialists, librarians, and 
corporate users from across the United States. 



3 . 2 POPULATION 

The literature review indicated that special libraries 
and corporations were among the first groups of potential 



^^Kai Arthur Sorensen, "A Delphi Analysis of the Gero- 
Communications Needs of the Aged in Marquette County, 
Michigan" (Ph.D. diss . , Kent State University, 1982) , 192; and 
Donald Ralph Cooper, "A Delphi Analysis of Priority Societal 
Problems and Resources of the Communication Field" (Ph.D. 
diss., Kent State University, 1977), 102-161. 
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users to extensively utilize the variety of information 
resources on the Internet. Early and continued use of the 
network by these groups gave them the advantage of experience 
and the rationale for choice of subjects for this study. After 
receipt of the e-mail message indicating an interest to 
participate, a preliminary form was sent to respondents which 
sought to narrow the group by expertise. Questions pertaining 
to professional background and knowledge of the Internet were 
asked and the final group consisted of twenty-five information 
specialists, librarians and corporate users. 

3 . 3 DATA COLLECTION AMD INSTRUMENTATION 

To date, there were no, research reports published in 
this area of interest. Therefore, the initial and subsequent 
questionnaires utilized were based on a combination of factors 
found to be important when evaluating other types of 
information resources. It was the objective of this study to 
increase the usefulness of the Internet for any potential user 
as well as provide a basis for further research in this area. 

The initial identification of participants was obtained 
by contact through e-mail and all subsequent questionnaires 
were sent through traditional mail service. It was felt that 
time spent answering questionnaires on-line might pose work- 
related problems. Also, many network administrators strongly 
discourage the use of their space for questionnaires 
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studies. The initial call for participants was posted to the 
following discussion and Usenet groups: 

1. Slaite-L - the Special Libraries discussion group 
originating at Babson College (address Slaite- 
L@BABSON.EDU) 

2. Buslib-L - the business discussion group (address 
Buslib-L@IBDSU.EDU) 

3. Libref-L - library reference and discussion group 
( address Libref -L@KENTViyi . KENT . EDU) 

4. Usenet group Biz . Comp . Services - business and computer 
related topics. 

5. Usenet group Biz. General - business topics in general. 

6. Usenet group Biz.Misc. - any topic related to 
business . 

The items included sought to identify areas of use on the 
Internet, information needs and current evaluation techniques. 
It was the intention of this study to gather quantitative data 
obtained through the Delphi rounds and "bring into focus the 
priorities and preferences of experts in positions to 
influence the development of the subject under 
consideration."^® Following analysis of the preliminary 
return, experts were selected based on their )<:nowledge and 
understanding of the Internet , as well as their use and amount 

^®Adele E. Friedrich, "Competencies for the Information 
Professional in the Coming Decade: A Delphi Study" (Ph.D. 

diss.. University of Pittsburg, 1985), 52, citing Elizabeth 

Dyer, "The Delphi Technique in Libraiy Research," Library 
Research 1 (1979) : 41-52. 
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of experience. A letter of invitation and outline of the 
study was then sent to qualified participants. Problems or 
questions regarding the questionnaires or study were answered 
through e-mail, but all rounds were sent through regular mail. 
The first round questionnaire was pre-tested on available 
students and faculty in the library and information science 
fields at Kent State University. Due to the small, homogeneous 
nature of the expert population, this study was reduced to two 
Delphi rounds . 

3.4 DESIGN 

Weeks 1 and 2-- Finalized literature review. 

Weeks 3 and 4 -- Developed and distributed introductory 
letter and preliminary questionnaire, pre-tested round one. 

Weeks 5,6,7 -- Posted notice through Internet for 

participants, sent out preliminary forms and selected experts. 

Weeks 8,9 -- Distributed round one. 

Weeks 10- 32 -- Analyzed results from first questionnaire 
and developed second questionnaire. 

Weeks 33 and 34 -- Distribute second questionnaire and 
analyzed results. 

Weeks 35,36,37 -- Analyzed final round. 

Week 38 -- Report findings. 

All questionnaires were de /eloped and revised by the 
researcher, in conjunction with respondents' input. The Lime 
schedule presented was not planned, but the result of a 
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variety of unforeseen factors. It is recognized that the delay 
between rounds one and two might be a limitation of this 
study, although a complete summary of round one was included 
with the final round. Data analysis was accomplished by 
utilization of the SPSS statistical program. The values 
assigned to each response along with the frequencies were 
analyzed. 

Preliminary Ro\ind. This questionnaire was constructed in 
order to better select on a scientific basis which of the 
interested professionals could best be considered "expert." 
(See Appendix B) This selection process is where the Delphi 
study again receives criticism. However, by selecting 
participants on the basis of their experience and knowledge, 
the impact of this criticism is lessened. The categories 
developed for this process were: Experieiice ; Services 
Performed; E-Conferences, Journals Used or Moderated; and, 
Type of Use. Additionally, a section for User Identification 
and Position was included in order to identify the 
characteristics of the population. Except for the Type of Use, 
all questions were closed format. The composition of the 
experts by position is shown in Table 1. 

Table 1 




Professional Position 



Special Librarians 



# of Final Experts 






1 
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University/ Academic Related 
Librarian 

Information Specialist 
Combination 



5 

6 
2 

11 



O 

ERIC 



Note - Combination was typically one of the categories 
mentioned above in addition to one or more of the other 
categories listed as choices. 

Of almost sixty individuals who responded to the initial e- 
mail posting (See Appendix A) and were sent preliminary 
packets, thirty-eight returned their preliminary packets. Of 
those thirty-eight, thirty experts were identified for the 
study. First round packets were then distributed. 

Delphi Roxmd One. A letter of invitation and outline of 
the study accompanied round one and was sent to thirty 
experts. (See Appendix C) A stamped, self-addressed envelope 
was included with each packet, and instructions to complete 
the forms. Respondents were, asked to return the completed 
forms within one week. A total of twenty-five questionnaires 
was returned. The questionnaire was designed in three separate 
sections; Inf ormation- -User Evaluation and Selection; Current 
Internet Evaluation; and Internet Resources- -Use and 
Organization. Each section contained a series of statements 
which were to b(? evaluated from the perspective of business 
usage, on a five-point Likert scale. Respondents were asked to 
rate their feelings from "Strongly Agree," "1" to "Strongly 
Disagree," "5." All questions v/ere worded in a positive 
manner, so a response for agree or disagree would be uniformly 
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rated. The total number of items in round one was thirty-six. 
A blank section was included at the conclusion of the 
questionnaire for additions or comments. 

Delphi Round Two. This round was designed to finalize 
consensus regarding evaluative strategies for business 
resources by taking into consideration results from round one. 
The twenty- five experts from round one were sent a cover 
letter with instructions for round two, summaries of round 
one, the second and final questionnaire, and a self-addressed 
stamped envelope. The time span allotted for return of this 
round was again one week. The questionnaire was designed with 
a five-point Likert scale response identical to round one. 
This round listed thirty suggestions or solutions to aid in 
the development of evaluative guidelines. Experts were again 
given an opportunity to add their comments or suggestions at 
the conclusion of the questionnaire. Due to the high rate of 
agreement within round one, the decision was made to drop the 
third round of this study. Rather than ranking a list and then 
rating the desirability, the second round offered both 
possible solutions and choice as to desirability in one step. 

3.5 DATA ANALYSIS - ROUND ONE 

The first round questionnaire was analyzed for frequency 
and weights. All results for round one can be found in 
Appendix D.) The weights were equivalent to the number given 
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on the scale, that being "Strongly Agree, 1" equal to "1" and 
"Strongly Disagree, 5" equal to "5." In cases where a question 
was not answered by all respondents, the average for that 
question was adjusted to match the number of actual responses. 
The mean score was determined for each item and only those 
items with an average of 2.0 or higher, or 3 . 0 and lower were 
retained for inclusion in the second round. It was felt that 
this would place a focus for the research in those areas in 
which experts had a strong commitment in either a positive or 
negative opinion. A few items from round one rated on the 
extreme. The following items in Table 2 received a high rate 
of agreement in current information use and evaluation 
(numbers correspond to item location in questionnaire) : 

Table 2 



Issue 



Mean Score 



A1 . Credibility of the provider 1.28 
A2 . Depth or scope 1.72 
A3. Accuracy 1.24 
A4 . Consistency/Convenience 1.80 
A5 . Clarity/Readability 1.76 
A6 . Currency 1.76 
A7 . Ability of searcher to understand 1.91 
A8 . Ability of users to understand 1.88 
A13 . Ease of access 1.72 



Of the Internet specific statements. Table 3 illustrates 
those which rated strong in agreement: 
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Table 3 



Issue 



Mean Score 



B2 . Evaluation of material is important 
B4 . Evaluation service would increase 
usefulness of the network 

B6 . Use of the network to meet information 
needs 

B9 . Use of the Internet increases work efficiency 
Cl. Use discussion groups, or Usenet to help 
decipher worthwhile resources 

C2 . Archie/Veronica useful for finding resources 
C3 . Gopher/Mosiac useful for finding resources 
C6 . Use of "readme" file to help evaluate 



2 . 00 



2 . 00 
2 . 00 



1.56 



1.96 
1 . 80 
1.36 
1 . 80 



The high degree of consensus and the utilization of standard 
evaluative techniques for business information indicated that 
this area of information usage did not require unique 
guidelines for evaluation. Reliance upon standard guidelines 
indicated that perhaps the second round should offer solutions 
to Internet evaluation by a combination of new and existing 
guidelines. These results were considered when constructing 
the second round statements. The scores for Internet specific 
items indicated that currently, peer evaluation was used 
extensively for evaluation and structured methods such as a 
file name do not represent the contents accurately, nor are 
they used in that manner. 

Of the first round statements which resulted in an 
overall consensus of disagreement, the following statements 
were of particular interest: 
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Table 4 



Issue 



Mean Score 



B8 . Utilization of the Internet for primarily 3.60 

e-mail 

C4 . File names accurately represent data 3.88 

contained within the files 

C5 . Judge relevance of information contained 4.08 

within the file by file names 

Cl. Indexes accurately reflect database contents 3.48 

Again, the methods which many researchers might use to quickly 
evaluate, those being file names or indexes, experts agreed 
were not accurate or useful for evaluation. Of interest is the 
expansion of Internet usage beyond primarily e-mail, 
indicating that this user population is putting more emphasis 
on the Internet as a primary information resource. With these 
results in mind, the second round aimed to provide solutions 
to these problems, or to offer suggestions which incorporated 
traditional evaluation guidelines within an electronic 
environment . 

3 . 6 DATA ANALYSIS - RODUD TWO 

Of the twenty- five questionnaires sent out, eighteen were 
returned. One of the eighteen was returned unanswered, 
dropping the respondents to seventeen. A tew possibilities 
exist as to reasoning behind the non- respondents . Primarily, 
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it may have been the length of time between rounds one and 
two, due to unforeseen problems. Secondly, positions and 
people change over time and it may be that some of the seven 
non-respondents had moved since the first round. Seventeen, 
however, was sixty-eight percent of the first round 
respondents, and was decided to be sufficient. 

Data analysis was performed using the SPSS statistical 
package. The available choices for responses were identical to 
those used in the prior round. Scores were averaged and the 
mean was based on the number of viable respondents. There were 
only three cases in which the questions were not completed by 
all respondents, and for those questions, the average was 
based on the number of complete responses. There are many 
discussions, articles, and overviews of the Internet and 
research. To date, the researcher could not locate any actual 
research dealing with these particular issues, aside from 
those mentioned previously. It is therefore recognized that 
the choices and composition of the surveys could be improved, 
and it is hoped that future research will note the possible 
flaws and strive to improve upon the results. 

Data from the second round met with greater consensus on 
both agreements and disagreements (see Appendix G) . 

Those items which gained highest agreement are shown in Table 

5 . 
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Table 5 



Issue 



Mean Score 



1. Include statements regarding the source 

of data 1.17 

2. Prefatory materials should be added 1.41 

8. Frequency of updates should be included 1.58 

13 . Directory organization varies from site to 
site 1.88 

20. Need to demonstrate usefulness of 

network beyond e-mail 1.52 

23 . Knowledge in Veronica queries would 
increase effectiveness of network use 1.81 

25. Better estimation of file contents 

by thesaurus construction 1.70 

26. Associate subject headings with files 1.82 

27. Recommended sites have "readme" files 1.68 

28. Greater detail in indexes 1.86 

30. Use of gopher/bookmarks recommended 1.52 



From these results, certain observations are evident. 
Currently, organization and evaluation of Internet sites need 
to be improved. Experts agree that usage has increased beyond 
the scope of e-mail, and they are looking for ways to organize 
and utilize this resource. The expansion of the World Wide Web 
and windows-based front-ends has created easier access, out 
has not addressed quality. Consideration of these results 
would indicate that to become a useful lousiness source, 
providers need to address those issues which researchers are 
missing. Greater details for indexes, readme files, update 
notifications, and source of the data are items which need to 
be improved or added. 
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Items which met with greatest disagreement are shown in 
the following table: 



Table 6 



Issue 



Mean Score 



11. Anyone can successfully navigate 

through a search 3.82 

15. Panel of catalogers should organize 
business sources 3.29 

18. Internet sources can replace print sources 3.05 

Of particular interest is the result from the question 
regarding catalogers . Respondents were in agreement that 
subject headings and greater detail in indexes would aid in 
the ability to locate appropriate documents. Yet, they 
disagree that catalogers should begin to organize sources . 
Perhaps the term "catalogers" partially affected the 
responses, although it would seem these individuals are most 
familiar with subject headings. For future studies, it might 
be advisable to offer choices of occupations such as indexers 
or abstractors rather than one particular area. The other 
results align with the sections that met with agreement. It 
seems evident that network organization does not yet allow for 
successful search results without aid from an information 
specialist, nor can this resource replace traditional print 
sources . 
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4. SIGNIFICANCE 



The opinions gathered from the experts in this study 
provided the first substantive research regarding network 
information resources and judgments of their quality and 
usefulness to special libraries and corporations. As the 
Internet grows in size and popularity, libraries and 
businesses can utilize the results of this research to 
increase their competencies in the electronic information age 
leading to expanded efficiency and viability. The results 
indicated the following statements should be considered when 
evaluating business resources on the Internet, or should be 
added to considerations when utilizing this resource: 

1. Look for materials which can provide aid to 
evaluation, such as prefatory materials and source of data. 
Reinforce the use of the Internet with secondary sources. 

2. The Internet is useful for more than just e-mail, but 
the wider user population within the business environment 
needs to understand the potential of this resource. At the 
same time, current organization of the Internet does not lend 
itself to successful search results without the aid of an 
information specialist. 

3. Currently, file names and directory names do not 
provide an accurate indication of the information contents. 
Further development into options such as thesaurus 
construction for file and database names, assigned subject 
headings, and standardized presentations were seen as options. 



but the feasibility may be questionable. Although no answer to 
this problem may be available, it is critical to evaluation of 
resources to ^understand the problems in these areas . 

Many of the questions which this study sought to clarify 
did not gain a significant response in either direction to 
enable a conclusion. There was a neutral position in regards 
to knowledge of evaluative guides currently available for the 
Internet. From many of the comments, there was an inability to 
distinguish locator guides from evaluative guides. From the 
researchers' view, a listing of available sources does not 
indicate an evaluative comment on those resources . Perhaps the 
wording of the statements could be revised in future studies 
to avoid this confusion. 

Neutrality was also the result from statements on groups 
which require certain standards before posting data. This area 
is in a constant state of flux, and perhaps the motion of the 
Internet will never be stilled enough to allow a statement 
such as this to find agreement. The movement of information 
and groups on the networks is such that many times, a location 
may be in existence for only a few months. However, knowledge 
of the nature of the networks can in itself be a method to 
evaluation. If the resource is heavily utilized and intact for 
a longer period of time, it may be open to more detailed 
analysis. 

The standards of importance to the business and special 
library communities seem evident from the results. The 

30 
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respondents represented a variety of backgrounds, and came to 
agreement on many of the issues facing information 
professionals today. It is hoped that these results will aid 
not only this group, but all those who seek to enhance their 
knowledge by sharing in the global community which has 
developed on the Internet. 
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August 24, 1994 

Dear Colleagues; 

I am a graduate student in the School of Library and 
Information Science at Kent State University. I am writing 
to solicit participants in an approved study which will help 
to develop evaluation guidelines for Business Resources on 
the Internet. Individuals from U.S. Special Libraries and 
Corporations are being considered as participants, even 
though emphasis is being placed on business resource 
evaluation . 

If you would like to volunteer to participate in this study, 
please send your e-mail and business address to the 
researcher: Kelly Heidman (kheidman@kentvm.kent.edu) 

Upon receipt of your e-mail message, a packet of materials 
detailing the study will be sent to you via regular mail. 
This invitation is the only part of this study which will 
be conducted via e-mail, barring unforeseen problems which 
may arise. 

There is no penalty of any kind if you choose not to 
participate in this study or if you would withdraw from 
participation at any time. Questions regarding this study 
may also be directed to the research advisor, Julie Gedeon, 
School of Library and Information Science, Kent State 
University, Kent, OH 44242; telephone: (216) 672-2782; 

e-mail: jgedeon@kentvm.kent.edu 

Thank you for your time and I look forward to hearing from 
you . 

Kelly Heidman 
Graduate Student 

School of Library and Information Science 
Kent State University 
Kent, OH 44242 

e-mail: kheidman@kentvm.kent.edu 
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38 




44 



RE: A Delphi Study to Ascertain Evaluation Guidelines for 
Business Resources on the Internet 

August, 1994 

Dear Colleague: 

This preliminary form has been sent in response to your e- 
mail message conveying an interest to participate in this 
study. I am a graduate student in the School of Library and 
Information Science at Kent State University. As part of the 
requirements for my master's degree I am conducting a study 
to develop evaluation guidelines for business resources 
available through the Internet. The enclosed questionnaire 
is designed to elicit information regarding your experience 
with the Internet. The responses will then be reviewed and 
experts chosen from the qualified pool. This study seeks to 
discover those qualities which special libraries and 
businesses deem important with regards to their information 
needs and the Internet. Ultimately, a set of benchmarks will 
be compiled against which these users can evaluate business 
resources available through the networks . 

Confidentiality and anonymity are guaranteed, as only the 
investigator has access to the survey data. For selection 
purposes, you will need to sign this preliminary 
questionnaire. There is no penalty of any kind if you should 
choose not to participate in this study 03 if you would 
withdraw from participation at any time. While your 
cooperation is essential to the success of this study, it 
is, of course, voluntary. A copy of the final results will 
be available upon request. 

If you have any further questions, please contact me at 
(216) 336-0108, e-mail KHeidman@Kentvm.kent.edu, or Julie 
Gedeon, my research advisor, at (216) 672-2782. If you have 

any further questions regarding research at kent State 
University you may contact Dr. Eugene Wenninger, Office of 
Research and Sponsored Programs, at (216) 672-2851. 

Thank you vez-y much for your cooperation and time; it is 
much appreciated. You may return the questionnaire in the 
enclosed self-addressed stamped envelope. 

Sincerely . 

Kelly Heidman 
Graduate Student 
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PRELIMINARY DATA 

Please indicate your responses to the following 
questions by circling the ' correct choice for your 
circumstance. Please call the researcher at (216) 336-0108 or 
send e-mail to KHEIDMAN@KENTVM.KENT.EDU if you have any 
questions regarding this questionnaire. 

USER IDENTIFICATION 

1. I am currently using the Internet in a special library 
setting . 

The user is working in one of the libraries classed in 
the subject index of the SLA Membership Directory. (See 
attached Note 1) 

2. I am currently using the Internet in a corporate setting. 

The user is working in a business setting but the 
information center or place of use is not officially labelled 
as a library by the corporation. 

3. Other-please explain. 



EXPER I ENCE /BACKGROUND 

Please indicate how you view your professional position by 
circling the appropriate choice. 

1. Information Specialist 

2 . Librarian 

3. Chief Information Officer 

4. Information Scientist 

5. Corporate Executive 

6. Information Broker 

7. Systems Analyst 

8 . Consultant 

9 . Database Manager 

10. Market Researcher 

11. Combination of any of the above 
numbers indicated above) . 

11. Other- -please specify 
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Please select tne most appropriate response to the following 
questions dealing with your experience and the Internet . 

1. Experience: 

I have used or been involved with the Internet 

a. More than five years. 

b. More than two years but less than five years. 

c . One to two years . 

d. Less than one year. 

2. Services performed (FTP, Gopher, Etc.) : 

I utilize FTP, Gopher, Telnet, or similar services 

a. More than twice per day. 

b. Once or twice per day. 

c. Once or twice per week. 

d. Once per month. 

3. E-Conferences, journals used or moderated: 

I subscribe to and/or moderate 

a. More than one e-conference or journal. 

b. Only one conference or journal. 

c. I do not subscribe to or moderate any journal or 
conference . 

4. Type of use (Research, E-Mail, Etc.) 

Please list how you typically use the Internet 
currently . 



List here any issue which you feel might be important for 
identification of the expert panelists that was not included 
in the questions above. 
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SPECIAL LIBRARY DESIGNATIONS- -Classed by subject^" 



NOTE 1 

1. Accounting 

2. Acid Rain 

3 . Advanced Measurement 

4 . Advertising 

5 . Aeronautical/ 

Aerospace Propulsion 

6 . Aerospace 

7 . Agrochemicals 

8. Allied 

9 . Aluminum 

10. Anthropology 

11. Archaeology 

12. Art History 

13 . Astronomy 

14. Atlases 

15. Automation Vendor 

16 . Automotive 

17. Automotive Industry 

18. Banking 

19. Batteries 

20. Biochemistry 

21. Botany 

22. Business 

23. Business- -Japanese 

24. Carbon and Graphite 

25. Cement 

26 . Ceramics 

27. Chemical Engineering 

28. Chemical Processes 

29. Chemical Industries 

30. Chemistry 

31. Coatings 

32. College Bookstores 

33. Colloid/Surf ace Science 

34. Competitive Intelligence 

35. Computer Science 

36. Computers 

37. Concrete 

38. Consultant 

39. Corporate Fundraising 

40. Databases 

41. Die Casting 

42. Economics 

43. Electric Power 

44 . Electrical Engineering 

45. Electrochemistry 



46. Electromagnetic Fields 
47 . Electronic Engineering 

48. Electronic Materials 

49. Employee Benefits 

50. Emulsion Polymerization 

51. Energy 

52 . Engineering 

53 . Environment 

54 . Environmental Biology 

55. Environmental Education 

56 . Federal Case Law 

57. Ferralloys Industry 

58. Finance 

59. Flower Arranging 

60. Foundation Fundraising 

61. Gardening 

62. Gazetteers 

63 . Government Purchasing 

64. Guidance and Control 

65. Health 

66. Heat Transfer 

67. Herbs 

68. High-Temperature 
Ceramic Materials 

69. Horticulture 

70. Independent Contractor 

71. Information Systems 

72 . Information Technology 

73 . Inorganic Chemistry 

74 . Investment 

75. Iron Industry 

76. Landscape Architecture 

77 . Law 

78. Legal Practice 

79. Library Management 

80. Lighting Technology 

81. Management 

82 . Manufacturing 
Technology 

83 . Maps 

84 . Market Research 

85. Marketing 

86. Materials 

87. Materials Science 

88. Mechanical Engineering 

89. Medicine 



^■'SLA, "SLA Cleveland Chapter Membership Directory" 
(Cleveland, Oh: Sherwin-Williams Company, 1993), 80-91. 
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90. Metals 

91. Metals/Materials 

92. Museology- 

93. Natural Gas 

94. Natural Gas Utilization 

95. Natural History 

96. Newspaper Library 

97. Newspapers (domestic) 

98. Nondestructive Evaluation 

99. Nonferrous Metallurgy 

100. Nonprofit Management 

101. Nursing 

102. Ohio Case Law 

103. Optics 

104. Overhead and Underground 
Distribution 

105. Paints 

106. Paleontology 

107. Personnel Management 

108. Petroleum 

109. Philanthropy 

110. Photographs 

111. Physics 

112. Plastics 

113. Pole Line Hardware 

114. Polymer Chemistry 

115. Polymers 

116. Power 

117. Recruitment Advertising 

118. Recruitment Markets 

119. Recruitment Media 

120. Research and Development 

121. Rubber 

122. Rubber Business 

123. Rubber Chemistry 

124 . Rubber Technology 

125. Sales 

126. Science 

127. Semiconductors 

128. Simulators 

129. Space Communications 

130. Standards 

131. Steel Industry 

132. Strategic Planning 

133. Structural Mechanics 

134. Sulfur Chemistry 

135. Tax 

136. Technology 

137. Thermoplastic Elastomers 

138. Toxicity 

139. Toxicology 

140. Training 



141 . Transportation 

142. Water-Soluble Polymers 

143. Weapons Systems 

144 . Welding 

145. Zoology 
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Dear Colleague: 

I am a graduate student in the School of Library and 
Information Science at Kent State University. You have been 
selected as an expert for a study which will help develop 
evaluation guidelines for. business resources on the Internet. 
This investigation will be conducted in the fall using the 
Delphi method. 

Individuals from all special libraries, universities and 
corporations were considered for candidates, even though 
emphasis was placed on business resource evaluation. Further 
discussion of the rationale, methodology and time commitment 
are contained in the outline of the study which follows. 

I am aware that limitations of work, travel and other 
activities may conflict with this project, but the time 
commitments on your part are minimal. I believe the outcomes 
will compensate you for your time and effort. Your input is 
highly va?.ued, and I can assure you every effort will be made 
by the researcher to accommodate your schedule . 

Confident uality and anonymity are guaranteed as only the 
investigator has access to the survey data. For tabulation 
purposes, you will need to sign the questionnaires. There is 
no penalty of any kind if you choose not to participate in 
this study or if you would withdraw from participation at any 
time. While your cooperation is essential to the success of 
this study, it is, of course, voluntary. 

Please review the overview of this study contained in the 
outline, complete the enclosed first round packet, and return 
the packet in the self-addressed stamped envelope within one 
week of receipt. Questions may be sent to the researcher 
through the following phone number or e-mail address: (216) 
336-0108 or KHeidman@kentvm.kent.edu or you may contact Julie 
Gedeon, my research advisor, at (216) 672-2782. If you have 
any further questions regarding research at Kent State 
University you may contact Dr. Eugene Wenninger, Office of 
Research and Sponsored Programs, at (216) 672-2851. 

Thank you for your time and consideration. I look forward to 
hearing from you. 

Sincerely, 



Kelly Heidman 
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OUTLINE OF THE STUDY 



Project Title: 

A Delphi Study to Ascertain Evaluation Guidelines for 
Business Resources on the Internet. 

Rationale : 

The Internet is a group of interconnected networks 
offering great opportunities for network information 
resources. The very characteristics which are presented as the 
strengths of the network also illustrate the weaknesses. The 
amount of information available throughout the networks is 
growing exponentially, while quality of information has yet to 
be addressed. Information overload is traditionally handled by 
weeding out the inferior resources. To date, there are no 
guidelines established to evaluate these resources from the 
conventional standpoints regularly applied to print resources. 
Furthermore, there is evidence that the approaches used to 
evaluate print or other electron.ic forms of information will 
not be adequate for this medium. 

It is the intention of this study to acquire a consensus 
from experts as to the use and needs of businesses, special 
libraries and universities with regards to their information 
supply and the Internet. The identification of these needs 
will be combined with evaluation criteria to establish 
guidelines for identifying and assessing valuable resources on 
the Internet for these users. 

Methodology : 

A modified Delphi will be used to collect data for this 
study. Delphi is a procedure for soliciting opinions and 
collating ideas using a series of sequential questionnaires 
and summarized feedback. 

The Delphi has three rounds of questionnaires. The first 
round will ask you to identify your present situation relating 
to information needs, criteria for evaluation, and your use of 
Internet business resources. Round two will summarize the 
results, and ask for consensus on the additions obtained in 
round one. Round three will summarize the data presented and 
ask for a final synopsis of necessary elements to be included 
in the established set of guidelines. 

Expected Outcomes : 

The results will provide substantive research in the 
area of assessing Internet resources. The guidelines will 
enable efficient use of network resources and increase the 
competitive edge for the user groups. 

Time Consumption; 

The estimated time of completion for the first round 
should be 30 minutes. Second and third rounds may require one 
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hour each. The time period allotted, however, for return of 
the questionnaires is one week. 
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QUESTIONNAIRE #1 



INTERNET GUIDELINES: DELPHI ROUND 1 



The objective of this first questionnaire is to develop 
a list of statements which identify areas of use on the 
Internet, information needs and current evaluation techniques. 
Review of interdisciplinary literature was performed to obtain 
the issues represented in this round. An agreement scale, 
identical to the one presented below, will be used in this 
questionnaire . 



INSTRUCTIONS 



Please indicate by circling the number on the scale, as 
to whether you agree or disagree with the statement. 



strongly Agree 1 2 


345 Strongly Disagree 


1 = 


strongly Agree 


This issue is a vital consideration 
in information needs, use or 
evaluation . 


2 = 


Agree 


I agree somewhat that this ' issue is 
important, but it is not vital. 


3 = 


Neutral 


I am neutral on this issue. I am not 
sure if this is important at all. 


4 = 


Disagree 


I disagree that this issue is 
important for consideration. 


5 = 


Strongly Disagree 


I strongly disagree that this issue 
is important to include for 
consideration . 



A "Comments" section appears following the issues presented. 
Please make a notation in this column if you feel any issue is 
unclear, demonstrates a bias, or needs revision in any way. An 
"Additions" section appears following the "Comments" . List any 
revisions or additions to the list which you feel are needed 
but were not represented in this section. 

Please return your questionnaire in the envelope provided. 







A. INFORMATION- -USER EVALUATION AND SELECTION 



1 . 


Credibility of the information provider is a factor STRONGLY 

which affects my choice of resources. AGREE 


1 


2 


3 


4 


STRONGLY 
5 DISAGREE 


2 . 


The depth or scope of the information obtained 
affects my choice of resources. 


1 


2 


3 


4 


5 


3. 


The extent to which the information is accurate and 
valid affects my choice of resources. 


1 


2 


3 


4 


5 


4 . 


The ability for information to be obtained consistently 

and conveniently from one source affects my choice of resources. 


1 


2 


3 


4 


5 


5. 


The extent of the clarity and readability of the information 
affects my choice of resources. 


1 


2 


3 


4 


5 


6 . 


The currency of the information affects my 
choice of resources. 


1 


2 


3 


4 


5 


7 . 


The ability of the searcher to understand the information 
provided affects my choice of resources. 


1 


2 


3 


4 


5 


8 . 


The ability of the users (the clients) to understand the 
information provided affects my choice of resources. 


1 


2 


3 


4 


5 


9. 


The uniqueness of the material available affects my 
choice of resources. 


1 


2 


3 


4 


5 


10 . 


The uniqueness of the information resource affects my 
choice of resources . 


1 


2 


3 


4 


5 


11 . 


When searching for information, the possible reference 
value for future use is an important factor to consider. 


1 


2 


3 


4 


5 


12 . 


The extent to which information is available through 
more than one source affects my choice of resources. 


1 


2 


3 


4 


5 


13. 


The ease of access to information affects my choice 
of resources. 


1 


2 


3 


4 


5 


14 . 


Pertinent information to my search query is an important 
factor to consider when trying to solve an information need. 


1 


2 


3 


4 


5 


15. 


Exploring new possibilities for information provision is 
important when trying to solve an information need. 


1 


2 


3 


4 


5 



n. CURRENT INTERNET EVALUATION 

1. Altliough directories for location of files exist, STRONGLY STRONGLY 

1 know of no evaluative guides to resources that AGREE 12345 DISAGREE 

exist on the Internet. 

2. Evaluation of material available through the Internet 12345 

is importrinf. for it to be valuable to address my information 

needs . 

K 1 know of no groups or experts who scan and retrieve 12345 

f reci-access data for accuracy or completenes.s . 

4. A service that would provide expert evaluation of 12345 

rf';;ources on the Internet would increase the usefulness 
ol the network. 



My business oi libraiy would eon.sider paying an additional fee 1 2 3 4 5 

I oj qvjaljty ievu;w.s ol information pioducts and services 
on t he Internet . 



U. 1 u.se inf orm.it ion available on the Infc'rnet to meet many of l 2 3 4 5 

my information needs. 

. I use 1 nf oririiil it>n available through the Internet, without 12^45 

guiiMiit t «•:: ..f a«<*in ,icy <>i <-oinj» 1 i-t i*n« ■ s.'-; . and f oi a 

jjida » y nour.i- i support t lie inf oj mat ion. 

». I use t.h(‘ Inteinet piimarily for commun i c.U i on witli 12 t 4 5 

eol 1 e.Kjiic/; (e-nniil), and do not aeeess s]>ei*ifit: information 
I <*;vnii oe.c n:\ ih" Internet . 
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9. 


Using the Internet tor information sources, exchange or supply 
increases my work efficiency. 


1 


2 


3 


4 


5 


10 . 


Aside from commercial vendors, I know of no group that 
requires a standard of quality before information can 
be posted. (Excluding moderated lists) 


1 


2 


3 


4 


5 


11, 


Quantity of information is important for 
for my information needs. 


], 


2 


3 


4 


5 


12 . 


Information representing a diversity of viewpoints is 
important for my information needs. 


1 


2 


3 


4 


5 



C. INTERNET RESOURCES- -USE AND ORGANIZATION 



1. I use services like Usenet, or discussion 
groups to help decipher worthwhile resources. 



STRONGLY 

AGREE 12 3 4 



STRONGLY 
5 DISAGREE 



2. Finding tools such as Archie or Veronica are 

useful for finding information resources on the Internet. 



1 2 3 4 5 



3 . Access tools such as Gopher or Mosiac are useful 
for finding information resources on the Internet. 



1 2 3 4 5 



4. The file names representing files within directories 

accurately represent the data contained within those files. 

b. I judge the relevance of information contained 
in a file by the file name, 

6, I look for a "readme" file to help evaluate the 
contents of a particular database. 

7. Indexes of database contents accurately reflect 
the contents of a database. 



1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 



8. T browse the frequently asked questions (FAQs) to 12345 

see if any information is available regarding 
evaluation of current resources. 



9, Utilization of network information centers such as 12345 

InterNIC increases the probability of locating 
relevant, quality resources. 



COMMENTS ; 



ADDITIONS: 
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DATA 



57 



n 



09:19:51 



KENT STATE UNIVERSITY 



IBM 4381 R24 



VM/CMS 5 . 0 



1 0 DATA LIST FILE ='NETl DATA NOTABLE, 

20 /I ID 1-4 QUESl TO QUES36 5-184 
3 0 FREQUENCIES VAR ALL. 

OTHERE ARE 4,085,240 BYTES OF MEMORY AVAILABLE. 

THE LARGEST CONTIGUOUS AREA HAS 3,241,504 BYTES. 

OMEMORY ALLOWS A TOTAL OF 32,767 VALUES ACCUMULATED ACROSS ALL VARIABLES. 
THERE MAY BE UP TO 8,192 VALUE LABELS FOR EACH VARIABLE. 

106-DEC-94 SPSS RELEASE 4.1 FOR IBM VM/CMS 



09:19:51 



KENT STATE UNIVERSITY 



IBM 4381 R24 



VM/CMS 5 . 0 



OID 

0 

VALUE LABEL 









VALID 


CUM 


VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 


1 


1 


4.0 


4.0 


4.0 


2 


1 


4.0 


4.0 


8 . 0 


3 


1 


4.0 


4 .0 


12.0 


4 


1 


4.0 


4 .0 


16 . 0 


5 


1 


4.0 


4.0 


20.0 


6 


1 


4.0 


4.0 


24 .0 


7 


1 


4.0 


4 .0 


28 . 0 


8 


1 


4.0 


4 .0 


32.0 


9 


1 


4.0 


4.0 


36.0 


10 


1 


4 . 0 


4.0 


40.0 


11 


i 


4.0 


4 .0 


44 .0 


12 


1 


4.0 


4.0 


48.0 


13 


1 


4.0 


4.0 


52.0 


14 


1 


4.0 


4.0 


56.0 


15 


1 


4.0 


4.0 


60.0 


16 


1 


4 . 0 


4.0 


64.0 


17 


1 


4.0 


4.0 


68 . 0 


18 


1 


4.0 


4 . 0 


72 . 0 


19 


1 


4.0 


4.0 


76 .0 


20 


1 


4.0 


4.0 


80.0 


21 


1 


4 . 0 


4.0 


84 . 0 


22 


1 


4.0 


4.0 


88.0 


23 


1 


4 . 0 


4 .0 


92.0 


24 


1 


4.0 


4.0 


96.0 


25 


1 


4.0 


4 .0 


100.0 


TOTAL 


25 


100.0 


100,0 





VALID CASES 
106-DEC-94 



25 



MISSING CASES 



SPSS RELEASE 4 . 1 FOR IBM VM/CMS 



09 : 19:51 


KENT 


STATE UNIVERSITY 


IBM 4381 


R24 VM/CMS 5 . 0 


QUESl 










VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 


16 


64 . 0 


64.0 


64 . 0 






2 


7 


28.0 


28.0 


92.0 






3 


1 


4 . 0 


4.0 


96.0 






4 


1 


4.0 


4.0 


100,0 






TOTAL 


25 


100.0 


100.0 




VALID CASES 


25 


MISSING CASES 0 








QUES2 










VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 


11 


44 .0 


44 .0 


44 .0 






2 


10 


40.0 


40.0 


84.0 






3 


4 


16 . 0 


16.0 


100 . 0 






TO'I'AL 


25 


100 . 0 


100.0 




OVALID CASES 


25 


MISSING CASES 


0 






CUES 3 










VALID 


CUM 


VALUE LAUF.L 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 


2 0 


80 . 0 


80.0 


80 . 0 






2 


4 


Hi . 0 


16 . 0 


96.0 






] 


1 


4 . 0 


4.0 


100.0 






TOTAL 


25 


1 00 . 0 


100.0 





ERIC 



56 



52 




53 



VALID CASES 25 MISSING CASES 0 



OUES4 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 




10 


40.0 


40.0 


40.0 






2 




11 


44.0 


44.0 


84 . 0 






3 




3 


12 .0 


12.0 


96 . 0 






4 




1 


4 .0 


4 .0 


100 . 0 






TOTAL 




25 


100.0 


100.0 




VALID CASES 


25 


MISSING CASES 


0 








OUES5 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 




9 


36.0 


36.0 


36 . 0 






2 




11 


44 .0 


44 .0 


80.0 






3 




3 


12.0 


12.0 


92 .0 






4 




2 


8 . 0 


8.0 


100 . 0 






TOTAL 




25 


100.0 


100.0 




VALID CASES 


25 


MISSING CASES 


0 








CUES 6 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 




12 


48.0 


48.0 


48 . 0 






2 




8 


32.0 


32.0 


80 . 0 






3 




4 


16.0 


16.0 


96 . 0 






4 




1 


4 .0 


4 . 0 


100 . 0 






TOTAL 




25 


100.0 


100.0 




VALID CASES 


25 


MISSING CASES 


0 








CUES 7 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






0 




1 


4.0 


4.0 


4 . 0 






1 




6 


24 .0 


24 .0 


28 . 0 






2 




14 


56.0 


56.0 


84 . 0 






3 




4 


16 . 0 


16.0 


100 . 0 






TOTAL 




25 


100.0 


100.0 




VALID CASES 


25 


MISSING CASES 


0 








CUES 8 












VALID 


CUM 


VAIAIE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 




11 


44 .0 


44 .0 


44 . 0 






2 




7 


28.0 


28.0 


72 . 0 






3 




6 


24 . 0 


24 .0 


96 . 0 






4 




1 


4 . 0 


4 .0 


100.0 






'rOTAL 




25 


100.0 


100 . 0 




VALID CASES 


25 


MISSING CASES 


0 








DUES 9 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 




5 


20.0 


20 . 0 


20 . 0 






2 




1 1 


■14.0 


44 . 0 


64 . 0 










7 


2H . 0 


28 . n 


92 . 0 






4 




2 


8 . 0 


8 . 0 


100.0 






TOTAL 




2 b 


100.0 


100.0 




VALID CASES 


25 


MISSING CASES 


0 








DUES 10 












VALID 




VALUE LABEL 




VAL'Ji: 


l-KKi." 


:’KNCY 


PEkC'ENT 


PERCENT 


PERCENT 






1 






20.0 


20 . 0 


. 0 






2 




K 


12 . 0 


1? . 0 


62.0 






\ 




B 


12 . 0 


12 . 0 


B4 . n 






4 




3 


12 0 


12.0 


96.0 
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5 




1 


4 .0 


4 . 0 


100. 0 






TOTAL 




25 


100 . 0 


100 . 0 




VALID CASES 


25 


MISSING CASES 


0 








QUESll 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






0 




1 


4.0 


4 .0 


4.0 






1 




6 


24 .0 


24 .0 


28 . 0 






2 




9 


36.0 


36,0 


64,0 






3 




7 


28,0 


28 .0 


92 , 0 






4 




2 


8,0 


8,0 


100,0 






TOTAL 




25 


100,0 


100 , 0 




VALID CASES 


25 


MISSING CASES 


0 








QUESi2 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






1 




7 


28,0 


28,0 


28 , 0 






2 




7 


28 ,0 


28.0 


56 . 0 






3 




7 


28,0 


28 . 0 


84 . 0 






4 




4 


16,0 


16.0 


100. 0 






TOTAL 




25 


100,0 


100.0 




VALID CASES 


25 


MISSING CASES 


0 








QUES13 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






]. 




12 


48,0 


48 . 0 


48.0 






2 




8 


32,0 


32 . 0 


80.0 






3 




5 


20,0 


20 . 0 


100.0 






TOTAL 




25 


100,0 


100 . 0 




VALID CASES 


25 


MISSING CASES 


0 








QUES14 












VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT 


PERCENT 






0 




1 


4 ,0 


4 . 0 


4 . 0 






1 




18 


72,0 


72 . 0 


76 . 0 






2 




6 


24 ,0 


24 . 0 


100. 0 






TOTAL 




25 


100.0 


100 . 0 




VALID CASES 


25 


MISSING CASES 


0 








CUES 15 




























VALID 


CUM 


VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENr 


PERCENl' 






1 




9 


36.0 


36.0 


36 , 0 






2 




10 


40.0 


40.0 


76,0 






3 




3 


12,0 


12 .0 


88 . 0 






4 




3 


12,0 


12 . 0 


100 , 0 






TOTAL 




25 


100.0 


100,0 




VALID CASES 


25 


MISSING CASES 


0 








DUES 16 












VALID 


CUM 
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Dear Colleague: 

My apologies for the amount of time it has taken to get 
the second round of questions to you for my study "A Delphi 
Study to Ascertain Evaluation Guidelines for Business 
Resources on the Internet . " 

I have included a summary of the results of the first 
round which you may keep. Because of the homogeneous nature of 
the respondent population and the overall consensus of ratings 
for the majority of items, the second round will now be the 
final round. 

Included with the second questionnaire is an explanation 
of the round along with instructions. 

Please return the questionnaire within one week of 
receipt in the enclosed self-addressed stamped envelope. Thank 
you for your participation and I will be sending all 
participants final results and summaries. 



Sincerely, 



Kelly R. Heidman 




t)b 
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Summary of Round One 



The objective of the first questionnaire was to obtain your judgments as to current information 
evaluation and organizational tools/resources on the Internet as they pertain to business sources. A 
five point scale was used to measure your responses, ranging from "1 - Strongly Agree" to "5 Strongly 
Disagree . " 



Data analysis was performed by taking each response for every item and determining the "average 
score" for each statement. The mean score was determined by assigning a weight to each possible 
numbered response. 



Weights were as follows: 
Response 
1 
2 

3 

4 

5 



Weight 

1 

2 

3 

4 

5 



Total weights for each statement were divided by the number of responses for that item. If the item was 
not answered by all participants, it was divided by the number of those who did respond. 

Items which were retained in this round maintained an average score of 2.00 or higher, or 3.00 and 
lower. These statements identified those items which were had either strong agreement or strong 
disagreement . 



ROUND ONE DATA ANALYSIS 

ITEM AVERAGE SCORE 

Credibility of the information provider is a factor 

which affects my choice of resources. 1.28 

The depth or scope of the information obtained 1,72 

affects my choice of resources. 

The extent to which the information is accurate and 1.24 

valid affects my choice of resources. 

The ability for information to be obtained consistently 1.80 

and conveniently from one source affects my choice of resources. 

The extent of the clarity and readability of the information 1.92 

affects my choice of resources. 

The currency of the information affects my 1.76 

choice of resources. 

The ability of the searcher to understand the information 1.91 

provided affects my choice of resources. 

The ability of the users {the clients) to understand the 
information provided affects my choice of resources. 1.88 



The ease of access to information affects my choice 1.72 

of resources. 

Pertinent information to my search query is an important 1.25 

factor to consider when trying to solve an information need. 

Exploring new possibilities for information provision is 2.00 

important when trying to solve an information need. 



Evaluation of material available through the Internet 2.00 

IS important for it to be valuable to address my information 
needs . 

A service that would provide export evaluation of 1.56 

resources on the Internet would increase the u.sefulnesG 
of the network. 

I use information available on the Internet to meet many of 2.00 

my information needs. 

1 use the InLerncl piimarily tor eommunieat ion with l.f»0 

rollcagues U» inail), and flo not •icci-ia, :;pr*fific infonr.alion 
resources or. the Internet . 

Using the Internet lor inlormafion ;?ourc{.*f; , exchange or .supply 2.00 
inf:reasr5J my work efticirncy. 
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I use ser\''ices like Usenet, or discussion 1.96 

groups to help decipher worthwhile resources. 

Finding tools such as Archie or Veronica are 1.80 

useful for finding information resources on the Internet. 

Access tools such as Gopher or Mosiac are useful 1.36 

for finding information resources on the Internet. 

The file names representing files within directories 3.88 

accurately represent the data contained within those files. 

I judge the relevance of information contained 4.08 

in a file by the file name. 

I look for a "readme" file to help evaluate the 1.80 

contents of a particular database. 

Indexes of database contents accurately reflect 3.48 

the contents of a database. 



YOU MAY KEEP THESE RESULTS. RETURN ONLY THE QUESTIONNAIRE IN THE ENVELOPE. 
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QUESTIONNAIRE #2 

A Delphi Study to Ascertain Evaluation Guidelines for Business 
resources on the Internet . 

The objective of this second and final round is to reach 
consensus on a list of evaluative criteria and considerations 
which would be most useful for application to Internet 
business resources. This study is not designating one 
particular aspect of business resources, but aims to develop 
generalized guidelines which can be applied to any type of 
business-related information. 

Items appearing in this round are a combination of 
revisions from round one along with some of your additions. 
Original statements from round one were restated as possible 
solutions or suggestions. 

A rating scale appears to the' right of each item and is 
identical to that which was used in round one. Please indicate 
which response most closely matches your point of view by 
circling the appropriate number. The scale is as follows: 

EVALUATIVE SCALE: 

1 Strongly Agree 

2 Agree 

3 Neutral 

4 Disagree 

5 Strongly Disagree 

A space is provided at the end of the questionnaire for any 
additional comments or suggestions. 

When you have completed the questionnaire, please place it in 
the self-addressed stamped envelope which is provided and mail 
it within one week after receipt. Thank you again for your 
time and participation. 




< 



0 






Suggest ions/Solut ions 
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Strongly 

Agree 



1. A statement regarding the source of the data should 

be included with the information to establish a 12 

credibility factor. 

2. Prefatory materials relating database/f i le contents 1 2 

should be added to enable evaluation of depth/scope 

of materials. 

3. Business resources available through the Internet 1 2 

should be reinforced by a second source to assist in 

accuracy . 

4. A notification as to file availability should be visible 1 2 

at the home page of a server to better assist the searcher 

in obtaining materials consistently and conveniently from 
one source . 



5. Uniform presentation of database/f i le contents should l 2 

be established to aid the searcher in obtaining materials 

consistently and conveniently from one source. 

6. Data prefaced with a statement of intended audience should 1 2 

be included to aid the searcher with factors of readability. 

7. Statements of intended audience are not needed because it is 12 

the responsibility of the information professional to obtain the 
appropriate materials for each client. 

8. Fre(^ency of updates appearing with databases/files should 1 2 

be included to aid the searcher in evaluation of currency. 

9. A central location with online updates of database contents and 1 2 

intended audience should be established. 

10. A central location which contained materials on similar issues 1 2 

for a variety of intellectual levels should be established to 

aid searchers in provision of appropriate information. 

11. Presentation and current organization of business resources on the 1 2 

Internet is such that anyone in the business environment can 

navigate successfully through a search. 

12. Current organization of business resources on the Internet is such 1 2 

that an information specialist should be consulted to obtain 

the best search results. 

13. Directories/files containing business resources vary from site to 12 

site as to their organization and currently need to be evaluated 

on an individual basis. 



Strongly 

Disagree 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 



14. Directories would be more functional to evaluation if uniformly 
designed from site to site. 

15. A panel of catalogers specializing in electronic information 
organizational patterns should be formed to organize business 
sources on the Internet . 

16. Business sources on the Internet are best utilized for ready 
reference questions (i.e. quick, factual answers). 

17. Business sources on the Internet are best utilized as supplemental 
materials to established reference source.s. 

18. Internet business resources can replace traditional print 
resources . 

19. Tiadilional evaluative criteria (i.e. currency, authority, etc.) 
will b«' adequate for evaluation of Internet resources. 

20. Information specialists need to demonstrate the usefulness 

of the Internet beyond e-mail to their respective user populations. 

.‘1 AecejH .)( int.ern»‘l. ))usincss resources to all people within Ihr 
potirtJlial usei environment would increase my work efliciency. 

22. Full Int(unol access (including Usenet or discussion groups) 

.sliould be encouraged in any bu.sine;3s environment. 

2 1. Knowledge in "how hesi. to compose a Veronica query" would 
incrt-iuu* the of f (‘ct i veness of tliis tool. 



1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
12 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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24 . 


To aid in gopher usefulness, control and organization of gopher 
client/servers should be placed with an information specialist. 


1 


2 


3 


4 


5 


25. 


Thesaurus construction or standardized practices for file names 
would facilitate the estimation of file contents by file name. 


1 


2 


3 


4 


5 


26. 


Associating subject headings with file names would increase the 
ability of the searcher to judge the appropriate documents by 
file name. 


1 


2 


3 


4 


5 


27. 


Sites on the Internet which would be considered as "recommended" 
should have "readme" files for their directories. 


1 


2 


3 


4 


5 


28. 


Indexes for sites should be more detailed in their presentation 
to be of help in reflecting database contents. 


1 


2 


3 


4 


5 


29. 


Lists of accessible files available on the Internet which are 
published by individuals are useful for evaluation of materials. 


1 


2 


3 


4 


5 


30. 


Use of gophers and/or bookmarks is a recommended method of easy 
access to heavily used sites and materials. 


1 


2 


3 


4 


5 
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100 . 0 



70 



VALID CASES 


17 


MISSING CASES 


0 






QUES3 










VALID 


CUM 












VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT PERCENT 




1 


5 


29.4 


29.4 


29.4 




2 


4 


23 . 5 


23 . 5 


52 . 9 




3 


6 


35 . 3 


35.3 


88 . 2 




4 


2 


11 . 8 


11 . 8 


100 . 0 
















TOTAL 


17 


100 . 0 


100.0 


VALID CASES 


17 


MISSING CASES 


0 






QUES4 










VALID 


CUM 












VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT PERCENT 




1 


4 


23 . 5 


23 . 5 


23 . 5 




2 


7 


41.2 


41 . 2 


64 . 7 




3 


5 


29 . 4 


29 . 4 


94 . 1 




4 


1 


5.9 


5 . 9 


100 . 0 
















TOTAL 


17 


100 . 0 


100.0 


VALID CASES 


17 


MISSING CASES 


0 







QUES5 

VALID 

CUM 

VALUE LABEL VALUE FREQUENCY PERCENT 

PERCENT PERCENT 




76 



71 







1 


4 


23 . 5 


23 . 5 


23 .5 




2 


6 


35.3 


35.3 


58.8 




3 


6 


35.3 


35.3 


94 . 1 




4 


1 


5 . 9 


5.9 


100 . 0 
















TOTAL 


17 


100 . 0 


100 . 0 


VALID CASES 


17 


MISSING CASES 


0 






QUES6 










VALID 


CUM 












VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT PERCENT 




1 


1 


5 . 9 


5.9 


5 . 9 




2 


5 


29.4 


29.4 


35 . 3 




3 


8 


47 . 1 


4 7.1 


82.4 




4 


3 


17 . 6 


17.6 


100 . 0 
















TOTAL 


17 


100 . 0 


100.0 


VALID CASES 


17 


MISSING CASES 


0 






QUES7 










VALID 


CUM 












VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT PERCENT 




1 


2 


11.8 


11 . 8 


11 . 8 




2 


o 


17.6 


17.6 


29.4 




3 


6 


35 . 3 


35.3 


64 . 7 




4 


5 


29 .4 


29.4 


94 . 1 




5 


1 


5.9 


5 . 9 


100 . 0 













Er|c 77 



72 

TOTAL 17 100.0 100.0 

VALID CASES 17 MISSING CASES 0 



QUES8 










CUM 








VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


52 . 9 




1 


9 52.9 


52 . 9 


94 . 1 




2 


7 41.2 


41.2 


100 . 0 




4 


1 5.9 


5 . 9 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 100.0 

0 


100 . 0 


QUES9 










CUM 








VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


35 . 3 




2 


6 35.3 


35 . 3 


76 . 5 




3 


7 41.2 


41 . 2 


88 . 2 




4 


2 11.8 


11 . 8 


100 . 0 




5 


2 11.8 


11.8 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 100.0 

0 


100 . 0 


QUESIO 










CUM 








VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 






2 


6 35.3 


35.3 



35 . 3 



ERIC 






78 



73 



76 . 5 




3 


7 


41.2 


41.2 


94 . 1 




4 


3 


17 . 6 


17.6 


100 . 0 




5 


1 


5 . 9 


5.9 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 . 0 


100 . 0 














QUESll 












CUN 










VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


5 . 9 




1 


1 


5 . 9 


5 . 9 


17.6 




2 


2 


11 . 8 


11 . 8 


23 . 5 




3 


1 


5 . 9 


5.9 


70 . 6 




4 


8 


47 . 1 


47.1 


100 . 0 




5 


5 


29.4 


29.4 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 .0 


100 . 0 


QUES12 












CUM 










VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


5 . 9 




1 


1 


5 . 9 


5.9 


64 . 7 




2 


10 


58 . 8 


58.8 


82 .4 




3 


3 


17 . 6 


17.6 


100 . 0 




4 


3 


17 . 6 


17 . 6 






TOTAL 


17 


100 . 0 


100 . 0 



O 




74 



VALID CASES 



17 



MISSING CASES 



QUES13 

CUM 

VALUE LABEL 
PERCENT PERCENT 



35 . 3 
88 . 2 
94 . 1 
100 . 0 



VALID 

VALUE FREQUENCY PERCENT 



1 

2 

4 

14 



6 

9 

1 

1 



35 . 3 
52 . 9 
5 . 9 
5 . 9 



35.3 
52 . 9 
5 . 9 
5.9 



VALID CASES 



17 



TOTAL 17 100.0 100.0 

MISSING CASES 0 



QUES14 

CUM 

VALUE LABEL 
PERCENT PERCENT 



29 . 4 
58 . 8 

82 . 4 
100 . 0 



VALID 

VALUE FREQUENCY PERCENT 



1 

2 

3 

4 



VALID CASES 



17 



TOTAL 

MISSING CASES 



5 

5 

4 

3 



29.4 

29.4 

23 . 5 

17 . 6 



17 100.0 

0 



29.4 

29.4 

23 . 5 

17 . 6 



100 . 0 



QUES15 

CUM 

VALUE LABEL 
PERCENT PERCENT 



VALID 

VALUE FREQUENCY PERCENT 





75 



11 . 8 
35 . 3 
47. 1 
76 . 5 
100 . 0 



1 2 11.8 

2 4 23.5 

3 2 11.8 

4 5 29.4 

5 4 23.5 



TOTAL 17 100.0 

VALID CASES 17 MISSING CASES 0 



QUES16 



CUM 

VALUE LABEL 


VALUE 


FREQUENCY 


PERCENT PERCENT 


0 


1 


5 . 


5.9 


1 


1 


5 . 


11.8 


2 


4 


23 . 


35 . 3 


3 


8 


47 . 


82.4 


4 


3 


17 . 


100 . 0 


TOTAL 


17 


100 . 


VALID CASES 17 


MISSING CASES 


0 





QUES17 



CUM 

VAI.UE LABEL VALUE FREQUENCY 

PERCENT PERCENT 



17.6 
52 . 9 



1 3 17.6 

2 6 35.3 



3 



23 . 5 




81 



11.8 

23.5 

11.8 

29.4 

23 . 5 



100.0 



VALID 

PERCENT 

5 . 9 
5.9 

23 . 5 
47 . 1 

17.6 



100 . 0 



VALID 

PERCENT 

17.6 

35.3 



76 . 5 



4 



23 .5 



76 



4 4 23.5 23.5 

100.0 







TOTAL 


17 


100 . 0 


100 . 0 


VALID CASES 


17 


MISSING CASES 


0 






QUES18 










VALID 


CUN 












VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT PERCENT 




1 


1 


5 . 9 


5 . 9 


5 . 9 




2 


7 


41.2 


41 . 2 


47.1 




3 


2 


11 . 8 


11.8 


00 

00 

LD 




4 


4 


23 . 5 


23.5 


82.4 




5 


3 


17 . 6 


17 . 6 


100 . 0 
















TOTAL 


17 


100 . 0 


100 . 0 


VALID CASES 


17 


MISSING CASES 


0 






QUES19 










VALID 


CUM 












VALUE LABEL 




VALUE 


FREQUENCY 


PERCENT 


PERCENT PERCENT 




1 


2 


11.8 


11.8 


11 . 8 




2 


7 


41.2 


41.2 


52 . 9 




3 


2 


11.8 


11 . 8 


64 . 7 




4 


5 


29.4 


29.4 


94 . 1 




5 


1 


5 . 9 


5 . 9 


100 . 0 
















TOTAL 


17 


100 .0 


100 . 0 


VALID CASES 


17 


MISSING CASES 


0 







82 



77 



QUES20 

CUM 

VALUE LABEL 
PERCENT PERCENT 



76 . 5 
88 . 2 
94 . 1 
100 . 0 









VALID 


VALUE 


FREQUENCY 


PERCENT 


1 


13 


76 . 5 


76 . 5 


2 


2 


11 . 8 


11 . 8 


3 


1 


5 . 9 


5 . 9 


4 


1 


5 . 9 


5.9 



VALID CASES 



17 



TOTAL 17 100.0 

MISSING CASES ■ 0 



100.0 



QUES21 

VALID 

CUM 

VALUE LABEL VALUE FREQUENCY PERCENT 

PERCENT PERCENT 



17 . 6 
47 . 1 
82.4 
100 . 0 



1 

2 

3 

4 



3 17.6 17.6 

5 29.4 29.4 

6 35.3 35.3 

3 17.6 17.6 



VALID CASES 



17 



TOTAL 

MISSING CASES 



17 100.0 

0 



100 . 0 



QUES22 

VALID 

CUM 

VALUE LABEL VALUE FREQUENCY PERCENT 

PERCENT PERCENT 



35.3 
58 . 8 
70 . 6 



1 

2 

3 



6 35.3 35.3 
4 23.5 23.5 
2 11.8 11.8 




S3 



78 



4 529.4 29.4 

100 . 0 



TOTAL 17 100.0 100.0 

VALID CASES 17 MISSING CASES 0 



QUES23 












CUM 










VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY . 


PERCENT 


5 . 9 




0 


1 


5.9 


5.9 


47 . 1 




1 


7 


41 . 2 


41 . 2 


88 . 2 




2 


7 


41 . 2 


41 . 2 


100 . 0 




4 


2 


11.8 


11.8 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 . 0 


100 . 0 


QUES24 












CUM 










VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


11 . 8 




1 


2 


11.8 


11 . 8 


58 .8 




2 


8 


47 . 1 


47 . 1 


76 . 5 




3 


3 


17.6 


17.6 


94 . 1 




4 


3 


17.6 


17.6 


100 . 0 




5 


1 


5 . 9 


5.9 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 . 0 


100 . 0 




84 



79 



QUES25 

CUM 

VALUE ISABEL 
PERCENT PERCENT 



41 . 2 

88 . 2 
100 . 0 



VALID 

VALUE FREQUENCY PERCENT 



1 7 41.2 

2 8 47.1 

3 2 11.8 



41 . 2 
47 . 1 
11 . 8 



VALID CASES 



17 



TOTAL 

MISSING CASES 



17 100.0 

0 



100 . 0 



QUES26 

CUM 

VALUE LABEL 
PERCENT PERCENT 



35 . 3 
88 . 2 
94 . 1 
100 . 0 









VALID 


VALUE 


FREQUENCY 


PERCENT 




6 


35.3 


35.3 


► 


9 


52 . 9 


52 . 9 




1 


5 . 9 


5 . 9 




1 


5 . 9 


5 . 9 



VALID CASES 



17 



TOTAL 17 100.0 

MISSING CASES 0 



100 . 0 



QUES27 

CUM 

VALUE LABEL 
PERCENT PERCENT 



5 . 9 

47 . 1 

88 . 2 



VALID 

VALUE FREQUENCY PERCENT 



0 

1 

2 



1 5.9 5.9 

7 41.2 41.2 

7 41.2 41.2 



85 



80 



100 . 0 




3 


2 


11.8 


11 . 8 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 . 0 


100 . 0 


QUES28 












CUM 










VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


11.8 




0 


2 


11 . 8 


11.8 


29.4 




1 


3 


17.6 


17.6 


94 . 1 




2 


11 


64 . 7 


64 . 7 


100 . 0 




3 


1 


5 . 9 


5.9 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 . 0 


100 . 0 














QUES29 












CUM 










VALID 


VALUE LABEL 
PERCENT PERCENT 




VALUE 


FREQUENCY 


PERCENT 


11 . 8 




1 


2 


11 . 8 


11 . 8 


70 . 6 




2 


10 


58 . 8 


58 . 8 


88 . 2 




3 


3 


17 . 6 


17.6 


100 . 0 




4 


2 


11 . 8 


11 . 8 


VALID CASES 


17 


TOTAL 

MISSING CASES 


17 

0 


100 . 0 


100 . 0 



QUES30 




86 



VALID 



CUM 

VALUE LABEL 
PERCENT PERCENT 



VALUE FREQUENCY PERCENT 



52 . 9 
94 . 1 
100 . 0 



1 

2 

3 



9 52.9 52.9 

7 41.2 41.2 

1 5.9 5.9 



VALID CASES 



TOTAL 17 100.0 100.0 

17 MISSING CASES 0 
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